14.10 Insight 768 Poggio

gateway to understanding intelligence in brains and machines, to discovering how the human brain works and to making intelligent machines that learn from experience. What distinguishes nontrivial learning from memory is the ability to generalize: that is, to apply what has been learned from limited experience to new situations. Memory bears the same relationship to learning as a dry list of experimental measurements does to a predictive scientific theory. The key question addressed here — from the perspective of the visual and motor systems — is what are the brain mechanisms for such generalization? Imagine looking for the phone in a new hotel room. Your visual system can easily spot it, even if you have never seen that particular phone or room before. So, learning to recognize is much more than straightforward pixel-by-pixel template matching. Visual recognition is a difficult computational problem, and it is a key problem for neuroscience. The main computational difficulty is that the visual system needs to generalize across huge variations in the appearance of an object; for instance, owing to viewpoint, illumination or occlusions. At the same time, the system needs to maintain specificity; for example, to identify a particular face among many similar ones. A similar ability to generalize is key to motor learning. Consider practicing how to hit a tennis ball: having learned to play a specific shot, you must then be able to use it under new conditions, adapting to changes in the spin on the incoming ball, the speed and direction of your opponent’s shots, the position of your body with respect to the ball, and so on. No two shots can be exactly the same, requiring a generalization ability of our motor program that can involve the modulation of thousands of motor units in new, adaptive ways. In abstract terms, generalization is the task of synthesizing a function that best represents the relationship between an input, x, and an output, y — an image and its label, say, or a desired equilibrium position of an arm and the set of forces necessary for attaining it — by learning from a set of ‘examples’, xi , yi. In this formulation, the problem of learning is similar to the problem of fitting a multivariate function to a certain number of measurement data. The key point is that the function must generalize. Generalization in this case is equivalent to the ability of estimating correctly the value of the function at points in the input space at which data are not available — that is, of interpolating ‘correctly’ between the data points. In a similar way, fitting experimental data can, in principle, uncover the underlying physical law, which can then be used in a predictive way. In this sense, the process of learning distils predictive ‘theories’ from data; that is, from experience. The modern mathematics of learning gives a precise definition of generalization and provides general conditions that guarantee it. It also implies that the ability to generalize in the brain depends mostly on the architecture of the networks used in the process of learning, rather than on the specific rules of synaptic plasticity. (The latter are reviewed in this issue by Abbott and Regehr, page 796.) Here, we highlight a network architecture supporting the ability to generalize in the visual and motor systems. Neurons at various levels of the visual cortex are generally tuned simultaneously to multiple attributes; that is, they respond to a particular pattern of their inputs, and the frequency of the firing follows a ‘tuning curve’, with a maximum for specific values of each of the attributes (together representing an optimum stimulus for the neuron), such as a particular direction of movement and a specific colour and orientation (Fig. 1 shows tuning for specific object views, each characterized by many parameters; see review in this issue by Tsodyks and Gilbert, page 775). We describe how a linear combination of the activities of such neurons can allow generalization, on the condition that the tuning is not too sharp, and that the weights of such a linear combination are what changes during learning. We then turn to the motor system and show that a linear combination of neural modules — each module involving several motor neurons innervating a coherent subset of muscles which together generate a force field — is mathematically equivalent to the linear combination of tuned neurons described for the visual system. Finally, we propose that the necessarily broad tuning of motor and visual neurons might be based on a canonical microcircuit repeated throughout different areas of cortex.